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ABSTRACT 

The  JQRR  metrics  for  Information  Assurance  (IA) 
and  Computer  Network  Defense  (CND)  are  combined 
with  a  framework  based  on  defense  graphs.  This  enables 
the  use  of  architectural  models  for  rational  decision  mak¬ 
ing,  based  on  the  mathematical  rigor  of  extended  influ¬ 
ence  diagrams.  A  sample  abstract  model  is  provided, 
along  with  a  simple  example  of  its  usage  to  assess  access 
control  vulnerability. 

1.  INTRODUCTION 

With  the  advent  of  Network  Centric  Warfare,  Infor¬ 
mation  Assurance  (IA)  is  becoming  ever  more  important 
to  the  success  of  military  operations.  Reliable  and  secure 
IT  systems  are  vital  to  ensure  success  on  the  battlefield, 
and  precisely  because  of  this,  they  also  become  the  focus 
of  adversarial  attention. 

IA,  however,  is  a  complicated  function  of  many  dif¬ 
ferent  concepts  such  as  technical  countermeasures,  orga¬ 
nizational  policies,  security  procedures,  and  more.  Mea¬ 
suring  the  level  of  IA,  therefore,  is  a  non-trivial  exercise; 
making  rational  decisions  and  prioritizations  about  the  use 
of  scarce  resources  is  ever  more  so. 

To  efficiently  protect  computer  networks  and  the  in¬ 
formation  stored  in  them,  combatant  commanders  and 
combat  support  agencies  need  to  be  able  to  assess  the 
current  security  level  of  their  IT  systems  as  well  as  the 
security  level  after  improvements.  An  example  of  a 
framework  for  such  assessment  is  the  Information  Assur¬ 
ance  (IA)  and  Computer  Network  Defense  (CND)  Joint 
Quarterly  Readiness  Review  (JQRR)  Metrics  (Joint 
Chiefs  of  Staff,  2003),  which  provides  six  different  cate¬ 
gories  of  metrics,  used  for  readiness  assessments  of  US 
forces:  1.  Personnel,  2.  Training,  3.  Operations, 

4.  Technology  (equipment),  5.  Supporting  Infrastructure, 
and  6.  Intelligence. 

The  diversity  of  these  metrics,  and  similar  ones,  pos¬ 
es  problems  of  how  to  accurately  weigh  them  all  together 
into  a  coherent  picture  of  security.  An  even  more  pressing 
problem,  however,  is  that  all  assessment  metrics  are  a 


priori,  while  the  actual  threat  consequences,  of  course,  are 
a  posteriori  notions.  This  is  causal  uncertainty. 

Furthermore,  decision  makers  using  metrics  face  a 
second  kind  of  uncertainty,  viz.  whether  information  and 
indicators  collected  during  a  security  assessment  are  cred¬ 
ible.  Measurement  errors,  misunderstandings  and  delibe¬ 
rate  deception  all  challenge  the  credibility  of  the  assess¬ 
ment  result.  This  is  measurement  uncertainty. 

This  paper  describes  a  method  for  how  to  combine 
Bayesian  statistics-based  extended  influence  diagrams 
with  attack  graphs  and  countermeasures  into  an  IA  as¬ 
sessment  framework.  This  framework  is  able  to  take  both 
types  of  uncertainty  into  consideration. 

This  approach  allows  a  mathematical  handling  of  the 
uncertainty  regarding  both  what  countermeasures  are  in 
place,  and  how  well  they  contribute  to  thwarting  attacks. 
The  Bayesian  approach  allows  calculating  the  probability 
that  attacks  succeed  from  an  enterprise  architecture  mod¬ 
el.  The  framework  also  takes  uncertainties  of  the  security 
assessment  into  consideration.  Moreover,  using  the  ex¬ 
tended  influence  diagram  formalism,  the  expected  loss 
from  each  attack  can  be  calculated.  Scenarios  can  be 
compared,  allowing  more  informed  decisions  of  how  to 
optimally  use  the  available  IA  resources. 

1.1  Outline 

The  remainder  of  this  paper  is  structured  as  follows. 
Section  2  addresses  some  related  works  on  security  me¬ 
trics  and  puts  the  present  contribution  into  context.  The 
important  concepts  of  attack  trees  and  defense  graphs  are 
introduced  in  section  3,  whereas  the  extended  influence 
diagrams  used  for  probabilistic  modeling  are  introduced 
in  section  4.  Section  5  provides  a  simple  example  of  how 
to  use  the  theory  thus  formed  for  IA  analysis.  Section  6 
explains  how  the  preceding  theories  can  be  integrated  into 
a  single  abstract  model.  Section  7  summarizes  the  contri¬ 
bution,  while  section  8  concludes  the  paper. 

2.  SECURITY  METRICS 

Within  the  field  of  security  and  information  assur¬ 
ance  research,  substantial  efforts  have  been  devoted  to 
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methods  and  methodologies  to  rank,  score  and  measure 
security.  In  addition  to  direct  metrics,  such  as  “the  percen¬ 
tage  of  systems  updated  with  the  latest  patches”,  a  number 
of  more  elaborate  measurement  and  ranking  methods 
have  been  suggested.  Some  examples  are  the  weakest 
adversary  metric  (Pamula  et  al.  2006),  mean-time-to- 
compromise  metric  (Leversage  and  James,  2008),  robust¬ 
ness  strategy  (Arber  et  al.,  2000),  the  attack  surface  metric 
(Manadhata  and  Wing,  2005),  operational  readiness  me¬ 
trics  (Connolly,  2001),  and  the  system  vulnerability  index 
(Alex-Foss  and  Barbosa,  1995). 

IA  depends  on  the  interaction  of  processes,  proce¬ 
dures,  tools  and  people  (Henning,  2001).  One  of  the  con¬ 
clusions  from  the  Workshop  on  Information-Security 
System  Rating  and  Ranking  (Henning,  2001)  was  that  it 
will  not  be  possible  to  successfully  quantify  the  assurance 
present  in  a  system  using  any  one  single  security  metric. 
Consequently,  metric  frameworks  typically  suggest  mul¬ 
tiple  metrics  for  these  different  domains;  see  for  example 
NIST  SP  800-55  (Swanson  et  al.,  2003)  or  the  JQRR 
metrics  (Joint  Chiefs  of  Staff,  2003).  Checklist  methods 
based  on  standards  such  as  ISO  17799  is  another  common 
practice  to  include  the  many  facets  of  security.  The  me¬ 
thods  typically  provide  a  list  of  indicators,  but  do  not 
describe  how  to  combine  these  indicators  into  an  overall 
value  for  security.  Yet,  an  overall  indicator  on  security  is 
desirable,  and  methods  to  combine  different  metrics  in  a 
meaningful  way  is  a  subject  of  research.  Some  work  has 
also  been  devoted  to  combing  metrics  into  an  overall 
indicator,  for  example  (Weiss  et  al.  2005)  and  (Johansson, 
2005). 

However,  no  prior  work  has  described  how  to  com¬ 
bine  metrics  while  taking  into  account  both  causal  uncer¬ 
tainty  and  measurement  uncertainty.  This  paper  suggests 
a  method  for  doing  so  by  using  attack  trees  as  the  struc¬ 
ture  for  aggregating  values  related  to  security  into  a  single 
measure. 

3.  ATTACK  TREES  AND  DEFENSE  GRAPHS 

Attack  trees  are  a  graphical  notation  evolved  from 
fault  trees,  where  the  main  goal  of  an  attacker  is  depicted 
as  the  root  of  a  tree  (Schechter,  2004).  The  steps  to  reach 
this  goal  are  broken  down  into  sub-goals  of  the  attack 
through  “AND”  and  “OR”  relationships.  This  is  a  stan¬ 
dard,  intuitive  way  of  modeling  threats  and  security. 

Attack  graphs  can  easily  grow  extensively.  To 
represent  them  more  compactly,  Liu  and  Hong  (2005) 
have  used  Bayesian  networks  to  express  them  and  to 
calculate  the  probability  of  an  attack  against  computer 
networks  being  successful  based  on  vulnerabilities  within 
it.  These  “Bayesian  attack  graphs”  can  be  used  to  answer 
questions  about  the  current  security  status  and  facilitate 
comparison  with  previous  measurements,  but  does  not 


answer  questions  about  how  to  improve  the  security  sta¬ 
tus.  Bayesian  networks  have  also  been  used  together  with 
attack  trees  to  analyze  other  security  related  concepts,  for 
example  with  the  purpose  of  intrusion  detection  (Qin  and 
Lee,  2004). 


A  natural  extension  of  attack  graphs  is  to  include  not 
only  attacks,  but  also  countermeasures.  From  the  perspec¬ 
tive  of  the  system  owner,  this  amounts  to  adding  controll¬ 
able  elements  to  the  graph.  In  (Howard  and  LeBlanc, 
2003)  countermeasures  are  modeled  together  with  trees 
depicting  threats,  and  in  the  theses  by  Foster  (2002)  and 
Schechter  (2004)  countermeasures  are  included  in  the  tree 
structures.  The  concept  of  including  countermeasures  in 
the  tree  structure  has  also  been  used  in  (Bistarelli  et  al. 
2006),  to  create  something  called  “defense  trees”,  illu¬ 
strated  in  Figure  1.  Techniques  have  been  presented 
which  use  defense  trees  for  strategic  evaluation  of  securi¬ 
ty  investments  (Bistarelli  et  al.  2006),  modeling  strategic 
games  in  security  (Bistarelli  et  al.  2007b)  as  well  as  mod¬ 
eling  of  conditional  preference  of  defense  techniques 
using  conditional  preference  nets  (Bistarelli  et  al.  2007a). 
Defense  trees  (or  graphs)  has  also  suggested  together  with 
extended  influence  diagrams  for  security  assessments  in 
Sommestad  et  al  (2008)  and  Sommestad  et  al.  (2009). 
This  paper  builds  on  that  work  and  describes  how  defense 
trees  can  be  connected  with  measurement  frameworks  to 
create  an  aggregate  indicator  on  security. 


Figure  1.  The  defense  tree  concept,  from  (Bistarelli, 
2007a). 


4.  EXTENDED  INFLUENCE  DIAGRAMS 

Extended  influence  diagrams  are  a  powerful  model¬ 
ing  approach,  used  to  depict  and  analyze  complex  causal 
interplay  between  quantities  (Johnson  et  al.,  2007b). 
These  diagrams  may  be  used  to  formally  specify  enter¬ 
prise  architecture  analysis  (Johnson  et  al.,  2007a).  The 
diagrams  are  an  extension  of  influence  diagrams,  as  de¬ 
scribed  by  Shachter  (1986  and  1988)  which  in  turn  are  an 
enhancement  of  Bayesian  networks  (cf.  Neapolitan  (2003) 
and  Jensen  (2001)).  In  extended  influence  diagrams,  ran¬ 
dom  variables  graphically  represented  as  chance  nodes 
may  assume  values,  or  states,  from  a  finite  domain  (cf. 
Fig.  2).  A  utility  node  represents  a  desired  goal,  such  as 
“Information  confidentiality”.  The  meaning  of  the  utility 
node  can  be  further  defined  by  other  nodes  that  it  has  a 
definitional  relation  to.  Causal  relations  on  the  other  hand 
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capture  associations  of  the  real  world,  such  as  “the  train¬ 
ing  of  system  administrators  affects  network  security”. 

As  illustrated  in  the  example  diagram  of  Fig.  2,  Ex¬ 
tended  Influence  Diagrams  can  be  used  to  represent  de¬ 
fense  trees.  A  utility  node  can  be  used  to  represent  the 
consequence  of  successful  attacks  and  the  steps  required 
for  their  success  can  be  decomposed  into  a  number  of 
substeps.  Attack  steps  will  then  assume  the  state  “Suc¬ 
cess”  or  “Failure”,  depending  on  the  states  of  its  parents. 
The  states  of  countermeasures  influence  the  probability 
that  an  attack  will  be  successful.  Thus,  they  are  modeled 
as  causal  parents  to  the  attack  steps.  Finally,  depending  on 
the  scenario  chosen,  the  states  of  countermeasures  will 
differ.  This  can  be  represented  by  decision  nodes  that 
influence  the  state  of  countermeasures.  (Sommestad  et  al, 
2008). 


In  order  to  specify  the  joint  distribution,  the  respec¬ 
tive  conditional  probabilities  that  appear  in  the  product 
form  (1)  must  be  defined. 

P{Xx,...,X„)=f\p{Xi\Pa{Xi))  (1) 

1=1 

The  second  component  P  describes  distributions  for 
each  possible  value  xt  of  Xh  and  pa(Xj)  of  Pa(Xj),  where 
Pa(Xj)  is  the  set  of  parent  nodes  of  Xj.  These  conditional 
probabilities  are  represented  in  matrices,  here  forth  called 
Conditional  Probability  Matrices  (CPMs).  Using  a  Baye¬ 
sian  network,  it  is  possible  to  answer  questions  such  as 
what  is  the  probability  of  variable  A  being  in  state  Xj  giv¬ 
en  that  its  parents  Y  and  Z  are  in  states  y2  and  z,  (Y  =  y2 
and  Z  =  Z]).  An  example  of  a  Bayesian  network  with 
CPMs  representing  the  probabilities  of  success  in  various 
attacks  is  shown  in  Figure  2. 


Extended  influence  diagram  syntax 


Node  Type 


Relationship  Type 


Causal  Relation  Informational  Relation  Definitional  Relation 


Example  diagram 

^Security' 


Attack  goal  1 

Success 

Failure 

Utility 

-1000 

0 

Figure  2.  Syntactic  elements  of  extended  influence 
diagrams  and  a  simple  example. 


The  mathematical  rigor  describing  the  causal  rela¬ 
tions  is  that  of  Bayesian  networks.  A  Bayesian  network, 
B  =  ( G ,  P),  can  be  described  as  a  representation  of  a  joint 
probability  distribution,  where  G  =  (V,  E )  is  a  directed 
acyclic  graph  consisting  of  vertices,  V,  and  edges,  E.  P  is 
the  probability  distribution  over  the  states  of  the  variables 
associated  with  each  vertex. 


One  important  feature  of  the  Bayesian  formalism  is 
the  possibility  to  learn  from  previous  data  and  create 
powerful  statistical  models  for  accurate  IA  assessments. 
Since  extended  influence  diagrams,  as  opposed  to  mere 
Bayesian  networks,  include  the  notions  of  decision  and 
utility  nodes,  predicted  losses  from  successful  attacks  can 
be  included  in  the  models,  thus  enabling  a  more  holistic 
view  of  IA. 

4.1.  Tests  in  Extended  influence  diagrams 

In  Bayesian  networks  and  extended  influence  dia¬ 
grams,  entities  are  often  modeled  that  are  exceedingly 
difficult  to  assess  directly.  When  modeling  high  level 
architectural  concepts  such  as  information  assurance, 
system  availability,  etc.  there  is  rarely  a  single  gold  stan¬ 
dard  of  measurement.  In  the  models,  this  is  reflected  by 
the  use  of  tests.  Tests  can  be  done  at  different  abstraction 
levels.  At  the  lowest  abstraction  level,  it  is  often 
straightforward  to  define  and  measure  things  like  the 
percentage  of  computers  that  are  fit  with  antivirus  soft¬ 
ware.  At  a  higher  level,  one  might  interview  stakeholders 
about  things  such  as  the  overall  competence  of  system 
administrators,  and  skip  the  details  of  how  they  acquired 
this  knowledge. 


In  a  Bayesian  network,  the  vertices  denote  a  domain 
of  random  variables  Xj,.  ..,X,  ,,  also  called  chance  nodes.  In 
the  context  of  concrete  models,  each  chance  node  corres¬ 
ponds  to  an  attribute.  Each  chance  node,  Xh  may  assume  a 
value  x,  from  the  finite  domain  Val(Xj).  The  advantage  of 
the  graph  representation  is  that  it  provides  a  compact  way 
of  expressing  the  dependency  relations  between  the  ran¬ 
dom  variables,  i.e.  which  variables  are  conditionally  inde¬ 
pendent  given  other  variables.  Each  edge  denotes  a  causal 
dependency  between  its  nodes. 


A  common  feature  of  such  tests  is  that  they  do  not 
reveal  definite  truths.  Rather,  test  have  a  level  of  credibili¬ 
ty  that  can  be  taken  into  account  when  performing  the 
analysis. 

Formally,  a  test  of  a  variable  is  represented  as  a  node, 
and  the  causality  arrow  is  directed  from  the  variable  to  the 
test.  Thus,  as  it  should  be,  the  result  of  the  test  depends  on 
the  state  of  the  variable,  as  illustrated  in  Figure  3.  The 
states  of  the  test,  {th  t2},  by  definition  correspond  to  the 
states  {x/,  x2}  of  the  variable  as  illustrated  in  the  table  in 
Figure  3. 
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X 

Xi 

X2 

T 

ti 

P(tl  \Xi) 

P(ti  \x2) 

1 2 

P(t2\x1) 

P(t2\x2) 

Figure  3.  A  node  X  with  the  test  T  and  the  CPM  for  the 
test  node. 

In  the  table  in  Figure  3,  the  outcome  of  the  test  is  re¬ 
lated  to  the  actual  states  of  the  variable,  i.e.  a  model  of  the 
accuracy  of  the  test.  A  perfect  test  would  correspond  to  an 
identity  matrix  CPM.  Since  most  realistic  (and  interest¬ 
ing)  tests  are  less  than  perfect,  the  CPM  will  rarely  be  an 
identity  matrix,  but  rather  reflect  measurement  uncertain¬ 
ty. 


perimeter  with  traffic  without  being  cut  off;  she  can  flood 
the  server  with  bad  IP  packets  to  make  it  unavailable. 
Two  other  options  are  simply  to  turn  off  the  power  to  the 
server,  or  rename  it  into  something  else.  Decomposing 
this  further,  the  adversary  is  required  to  gain  access  to 
power  breakers  or  to  turn  off  the  power,  and  require  her 
access  to  the  computer  room  and  server  interface  to  re¬ 
name  it,  respectively. 

A  number  of  defense  measures  can  mitigate  this 
threat  or  at  least  make  the  steps  in  this  attack  more  diffi¬ 
cult  to  accomplish.  In  this  simplified  example,  antivirus 
software  and  patched  systems  will  provide  some  protec¬ 
tion  against  attacks  directed  towards  the  DNS  server. 
Having  this  functionality  at  the  web  server’s  local  DNS 
server  naturally  does  not  offer  protection  against  com¬ 
promised  servers  at  the  client  side.  However,  it  does  make 
attacks  against  the  server  side  more  difficult. 


5.  A  SERVER  ATTACK  EXAMPLE 

Figure  4  illustrates  a  defense  tree  for  spoofing  attacks 
directed  against  servers  using  the  Internet  Protocol  (IP), 
inspired  by  (Howard  and  LeBlanc,  2003).  Examples  of 
military  IP  based  networks  include  the  NIPRNet  and  the 
SIPRNet,  as  well  as  the  tactical  voice  over  IP  network 
RIPRNet.  A  spoofing  attack  will  require  the  adversary  to 
both  knock  out  the  valid  machine  and  at  the  same  time 
have  created  a  new  one  with  the  same  name.  Knocking 
out  a  server  can  be  done  in  four  ways.  Firstly,  the  attacker 
may  hijack  the  DNS  of  the  server  by  infecting  it  with 
malicious  code  or  exploit  some  other  vulnerability.  Se¬ 
condly,  if  the  adversary  is  able  to  bypass  the  network 


The  ability  to  resist  flooding  of  the  server  with  bad  IP 
packets  is  strengthened  by  using  proper  boundary  protec¬ 
tion  (firewalls)  and  the  use  of  intrusion  detection  systems 
(IDS)  that  can  alert  administrators  of  anomalies.  Access 
to  the  computer  room  as  well  as  its  power  can  be  re¬ 
stricted  using  physical  access  control  mechanisms.  Unau¬ 
thorized  access  to  the  server  can  be  mitigated  with  logical 
access  controls,  such  as  password  protection. 

This  example  illustrates  attack  vectors  and  counter¬ 
measures  for  an  attack  that  spoofs  a  content  server  on  an 
IP  based  network,  such  as  the  NIPRNet. 


Figure  4.  Example  defense  three.  Inspired  by  (Howard  and  LeBlanc,  2003).  The  attack  steps  are  the  controllable  nodes 
of  the  attacker,  while  the  security  variables  are  the  controllable  nodes  of  the  system  owner. 
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A  security  assessment  investigating  the  possibilities  of 
various  adversarial  attacks  will  assess  the  probability  of 
success  associated  with  the  different  attack  paths  in  the 
model,  as  remedied  by  the  associated  security  controls 
implemented.  The  JQRR  metrics  provides  in  total  82 
metrics  that  are  used  to  assess  the  level  of  IA  and  com¬ 
puter  network  defense  (CND)  of  DoD  information  sys¬ 
tems.  Some  of  these  are  a  posteriori  indicators  on  the 
historical  success  rates  of  hostile  attacks,  while  others 
indicate  the  current  state  of  countermeasures.  For  clarity 
and  brevity,  a  suitable  subset  of  these  metrics  has  been 
selected  for  the  purpose  of  the  present  example. 

5.1.  Incorporating  metrics  into  the  models 

As  described  in  section  4.1,  it  is  often  exceedingly  diffi¬ 
cult  to  directly  assess  complex  attributes  such  as  the  level 
of  information  assurance  at  a  military  facility.  However, 
more  low-level  attributes  of  security  facilities,  such  as  the 
percentage  of  networks  that  have  firewalls,  or  the  percen¬ 
tage  of  system  administrators  that  are  properly  certified 
can  usually  be  found.  Such  data  can  be  regarded  as  evi¬ 
dence  on  the  state  of  the  true  variables.  For  example, 
consider  the  competence  of  the  system  administrators 
employed.  The  competence  of  a  system  administrator  is  a 
non-tangible,  complex  attribute,  affected  by  a  number  of 
factors  such  as  general  experience,  previous  postings, 
formal  education,  certifications,  ability  to  work  under 
time  pressure,  etc.  Clearly,  these  cannot  all  be  modeled. 
What  can  be  done,  however,  is  an  assessment  based  on  a 
few  simple  attributes,  such  as  the  percentage  of  system 
administrators  that  are  certified  and  whether  regular  and 
proactive  vulnerability  analyses  are  carried  out. 

Table  1.  A  sample  CPM,  relating  the  competence  of 
system  administrators  to  a  measurable  variable. _ 


|  Competence  of  sysadmins 

High 

Low 

Regular  and  proactive 
vulnerability  analyses 

Yes 

0.95 

0.50 

No 

0.05 

0.50 

In  Table  1,  an  example  is  given  of  how  the  compe¬ 
tence  of  system  administrators  might  be  related  to  the 
existence  of  regular  and  proactive  vulnerability  analyses. 
It  is  reasonable  to  assume  that  such  analyses  occur  with  a 
very  high  probability  if  the  administrators  are  highly 
competent,  while  less  competent  administrators  will  not 
be  equally  heedful. 

Table  2.  A  sample  CPM,  relating  the  competence  of 
system  administrators  to  their  level  of  certification. _ 


|  Competence  of  sysadmins 

High 

Low 

Percentage  of  certified 
sysadmins 

Yes 

0.80 

0.30 

No 

0.20 

0.70 

Similarly,  Table  2  shows  how  the  competence  of  sys¬ 
tem  administrators  might  be  related  to  their  level  of  certi¬ 
fication.  Now,  assuming  that  these  fairly  straightforward 


relations  hold,  inference  about  the  abstract  and  more  elu¬ 
sive  competence  of  the  system  administrators  can  be  car¬ 
ried  out  using  Bayes’  theorem. 

5.2.  The  JQRR  metrics 

The  JQRR  metric  3.3.1  is  an  a  posteriori  indicator 
that  deals  with  incidents  of  unauthorized  access  during  the 
last  reporting  period.  Put  in  the  context  of  this  example, 
this  indicates  whether  the  attack  is  possible  or  not  and  it 
would  indicate  whether  access  can  be  gained  to  the  con¬ 
tents  server  or  the  DNS  server. 

Figure  4  further  includes  a  number  of  metrics  that  in¬ 
dicate  the  state  of  countermeasures.  The  percentage  of 
computers  with  antivirus  software  installed  (JQRR  4.1.3) 
provides  an  indicator  on  whether  the  DNS  server  has  such 
software  installed.  Metric  3.1.5,  measures  the  readiness  of 
regular  and  proactive  vulnerability  assessments  and  gives 
information  that  indicates  whether  systems  are  sufficient¬ 
ly  patched  and  updated. 

The  percentage  of  firewalls  that  are  installed  as  per¬ 
centage  of  the  number  of  required  ones  (JQRR  4.1.2) 
indicates  whether  the  network  is  firewall  protected.  Me¬ 
tric  2.1.1  is  the  percentage  of  system  administrators  that 
are  certified.  This  is  assumed  to  be  an  indication  of  the 
quality  of  the  network’s  firewall  protection  and  IDS. 

Physical  security  is  assessed  through  JQRR  metric 

5.2.2.  Logical  access  management  is  measured  through 
three  indicators:  (1)  The  number  of  system  administrators 
compared  to  the  number  required,  (2)  the  percentage  of 
users  that  has  passed  training  and  awareness  require¬ 
ments,  and  (3)  the  percentage  of  network  information 
management  personnel  compared  to  the  number  required. 
These  three  jointly  indicate  the  state  of  logical  access 
control  management. 

6.  ABSTRACT  MODEL 

The  preceding  example  sketches  but  one  out  of  many 
possible  attacks  and  uses  but  a  few  of  all  the  JQRR  me¬ 
trics.  In  order  to  generalize  this  example  and  employ  the 
method  proposed  on  a  broader  scale,  it  is  necessary  to 
approach  the  problem  in  a  more  abstract  fashion.  The 
concept  of  metamodels  helps  us  to  do  that. 

A  metamodel  is  a  collection  of  concepts,  used  as 
building  blocks  when  modeling  the  world.  A  metamodel 
formalizes  the  fact  that  certain  entities  and  relationships 
(e.g.  “computer”  and  “firewall”)  are  particularly  impor¬ 
tant  to  include  in  IA  assessment  models.  Having  identi¬ 
fied  these  concepts  and  relations,  they  can  be  used  as 
templates  in  practical  modeling  cases. 
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Figure  5.  A  metamodel  describing  the  entities  and  relations  used  in  the  defense  part  of  the  attack  example. 


The  use  of  metamodels  guides  modelers,  enforces 
coherent  practices  and  terminology,  and  enables  the  use 
of  underlying  theoretical  concepts.  Figure  5  illustrates  a 
metamodel  of  the  entities  and  relations  used  in  the  attack 
example  depicted  in  Figure  4.  As  is  readily  seen,  this 
metamodel  matches  the  example  and  the  domain  of  analy¬ 
sis  in  the  sense  that  a  modeler  who  uses  its  concepts  as 
her  building  blocks  will  inevitably  create  a  model  well 
suited  for  IA  analysis. 

The  usefulness  of  metamodels,  however,  is  even 
more  evident  when  considering  their  link  to  the  mathe¬ 
matical  formalism  embedded  in  the  extended  influence 
diagrams.  Each  attribute  relation  corresponds  to  a  proba¬ 
bilistic  effect  of  one  attribute  on  another,  and  to  a  CPM 
quantifying  this.  The  term  abstract  model  is  used  for  a 
metamodel  that  is  augmented  with  an  extended  influence 
diagram  describing,  in  a  Bayesian  fashion,  the  causal 
relationships  between  the  entities  involved  (Johnson  et  al, 
2007b). 

Detailed  guidance  for  modeling  of  DoD  systems  and 
operations  is  beyond  the  scope  of  this  paper,  but  can  be 
found  for  instance  in  the  DoD  Architecture  Framework 
(Department  of  Defense,  2007)  and  in  the  extensive  re¬ 
lated  literature. 

Using  the  metamodel  depicted  in  Figure  5,  concrete 
situations  can  be  modeled  and  assessed  with  respect  to  IA. 
In  Figure  6,  a  simple  situation  is  described,  using  entities 
and  relations  from  the  metamodel.  System  administrators 
Kevin  and  James  administrates  servers  (one  DNS  server 
and  one  NIPRNet  contents  server),  and  users  Douglas, 


Robert  and  Kim  all  use  either  the  contents  server  or  have 
access  to  the  building  where  it  is  located.  By  prescribing  a 
terminology  for  describing  this  situation,  the  metamodel 
facilitates  analysis  of  the  concrete  model  using  the  ex¬ 
tended  influence  diagram  formalism.  A  concrete  outcome 
of  such  an  analysis  might  be  a  32%  risk  of  server  spoof¬ 
ing,  entailing  an  expected  monetary  loss  of  $  2  million. 
Another  outcome  might  be  a  recommendation  on  how  to 
enhance  the  IA  level. 

7.  DISCUSSION 

The  method  described  in  the  previous  sections  pro¬ 
vides  a  framework  for  IA  analysis  with  a  number  of  nota¬ 
ble  strengths.  Firstly,  the  use  of  abstract  models  integrates 
the  use  of  existing  metrics  with  the  Bayesian  formalism. 

Secondly,  the  Bayesian  formalism  is  well  suited  to 
handle  both  causal  and  measurement  uncertainty,  thus 
making  the  most  of  each  IA  assessment.  Together  with 
historical  data  on  attacks,  this  facilitates  calculation  of 
expected  loss  for  both  the  current  state  of  systems  and 
potential  future  scenarios. 

Thirdly,  information  on  expected  losses  prior  to  and 
after  IA  improvements  enables  more  rational  decision 
making.  Using  the  framework  proposed,  combatant  com¬ 
manders  and  their  staffs  can  create  models  of  current  and 
potential  future  scenarios  based  on  metamodels  covering 
the  concepts  relevant  to  IA. 

It  is  worth  to  dwell  on  the  possibility  of  training  the 
underlying  Bayesian  network  using  historical  data. 
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Figure  6.  A  concrete  model  describing  entities  and  relations  that  could  be  involved  in  an  actual  IA  scenario. 


The  present  JQRR  IA  and  CND  metrics  have  been 
used  by  US  defense  services  and  DoD  combat  support 
agencies  since  July  2003.  This  means  that  a  lot  of  data  has 
been  amassed  and  can  be  put  to  use  within  an  extended 
influence  diagram  framework. 

The  training  of  Bayesian  networks  is  a  subject  exten¬ 
sively  treated  in  the  literature  (Jensen,  2001),  (Russell  and 
Norvig,  2003),  (Friedman,  1998).  Bayesian  networks  can 
be  trained  using  expectation-maximization  (EM)  algo¬ 
rithms,  which  in  simple  cases  essentially  reduce  to  the 
standard  Bayesian  inference  algorithm.  More  complicated 
cases  can  become  computationally  too  complex  and  thus 
require  methods  such  as  Markov  Chain  Monte  Carlo 
(MCMC)  for  approximate  Bayesian  learning.  An  example 
of  a  freely  available  software  tool  for  EM  learning  of 
Bayesian  networks  is  GeNIe,  developed  by  the  Decision 
Systems  Laboratory  at  the  University  of  Pittsburgh. 

An  extended  influence  diagram  that  has  undergone 
proper  learning  with  a  posteriori  measurement  data  be¬ 
comes  a  powerful  tool  to  assess  the  current  IA  level  of 
military  units  and  DoD  combat  support  agencies.  Fur¬ 
thermore,  it  provides  a  compact  and  intuitive  representa¬ 
tion  of  complex  dependencies  within  the  IA  domain, 
leading  to  increased  usability. 

8.  CONCLUSIONS 

The  present  paper  uses  the  JQRR  IA  and  CND  me¬ 
trics,  and  shows  how  their  use  can  be  extended  and  im¬ 


proved  within  a  framework  based  on  defense  graphs.  A 
sample  abstract  model  was  provided,  along  with  a  simple 
example  of  its  usage  to  assess  access  control  vulnerability 
of  an  IP  based  military  system  such  as  the  NIPRNet.  The 
prospects  for  training  a  probabilistic  inference  engine 
based  on  historical  data  were  discussed  and  identified  as  a 
potentially  powerful  method  for  making  more  rational  IA 
assessments,  a  key  factor  in  information  warfare. 
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